Living Labs for UMAP Evaluation
نویسنده
چکیده
Generating shared task initiatives in the user-modelling, adaptation and personalization (UMAP) space is difficult, especially given individual difference, privacy concerns and the interactive nature of the space. We put forward that the living labs evaluation paradigm, i.e., observing users in their natural task environments, has potential to overcome these difficulties and to allow for comparative evaluation in the UMAP space of research. In particular, the emerging approach to living labs for shared evaluation in other research disciplines, has potential to be adapted to allow for shared evaluation tasks in the UMAP community. Coupled with this, there is the potential to create living labs in different ways for shared UMAP evaluation. In this paper we overview the use to-date of living labs approaches for shared evaluation and set directions for its application in UMAP shared challenges.
منابع مشابه
Research-Centres Centred Living Labs
Developing means to conduct shared evaluation in the user modelling, adaptation and personalization (UMAP) space is inherently difficult. Not least because of privacy concerns and individual differences in behaviours between users of systems. In this paper we propose a research-centre centred living labs approach as one potential way to overcome these difficulties and to allow for shared task g...
متن کاملExperience in Evaluating AAL Solutions in Living Labs
Ambient assisted living (AAL) is a complex field, where different technologies are integrated to offer solutions for the benefit of different stakeholders. Several evaluation techniques are commonly applied that tackle specific aspects of AAL; however, holistic evaluation approaches are lacking when addressing the needs of both developers and end-users. Living labs have been often used as real-...
متن کاملMobile Living Labs 09: Methods and Tools for Evaluation in the Wild
In a Mobile Living Lab, mobile devices are used to evaluate concepts and prototypes in real-life settings. In other words, the lab is brought to the people. This workshop provides a forum for researchers and practitioners to share experiences and issues with methods and tools for Mobile Living Labs. In particular, we seek to bring together people who have applied methods for Mobile Living Labs ...
متن کاملTowards a Living Lab for Information Retrieval Research and Development - A Proposal for a Living Lab for Product Search Tasks
The notion of having a “living lab” to undertaken evaluations has been proposed by a number of proponents within the field of Information Retrieval (IR). However, what such a living lab might look like and how it might be setup has not been discussed in detail. Living labs have a number of appealing points such as realistic evaluation contexts where tasks are directly linked to user experience ...
متن کامل